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Abstract — There has been considerable recent work developing 
a new stochastic network utility maximization framework using 
Backpressure algorithms, also known as MaxWeight. A key open 
problem has been the development of utility-optimal algorithms 
that are also delay efficient. In this paper, we show that the 
Backpressure algorithm, when combined with the LIFO queueing 
discipline (called LIFO-Backpressure), is able to achieve a utility 
that is within Oil/V) of the optimal value for any scalar 
V > 1, while maintaining an average delay of 0([log(V)] 2 ) 
for all but a tiny fraction of the network traffic. This result 
holds for general stochastic network optimization problems and 
general Markovian dynamics. Remarkably, the performance of 
LIFO-Backpressure can be achieved by simply changing the 
queueing discipline; it requires no other modifications of the 
original Backpressure algorithm. We validate the results through 
empirical measurements from a sensor network testbed, which 
show good match between theory and practice. 

Index Terms — Queueing, Dynamic Control, LIFO scheduling, 
Lyapunov analysis, Stochastic Optimization 



I. Introduction 

Recent developments in stochastic network optimization 
theory have yielded a very general framework that solves a 
large class of networking problems of the following form: We 
are given a discrete time stochastic network. The network state, 
which describes current realization of the underlying network 
randomness, such as the network channel condition, is time 
varying according to some probability law. A network con- 
troller performs some action based on the observed network 
state at every time slot. The chosen action incurs a cost, F] 
but also serves some amount of traffic and possibly generates 
new traffic for the network. This traffic causes congestion, and 
thus leads to backlogs at nodes in the network. The goal of the 
controller is to minimize its time average cost subject to the 
constraint that the time average total backlog in the network 
be kept finite. 

This general setting models a large class of networking 
problems ranging from traffic routing [1J, flow utility max- 
imization [2], network pricing |3) to cognitive radio applica- 
tions [4|. Also, many techniques have also been applied to 
this problem (see [5| for a survey). Among the approaches 
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that have been adopted, the family of Backpressure algorithms 
|6| are recently receiving much attention due to their provable 
performance guarantees, robustness to stochastic network con- 
ditions and, most importantly, their ability to achieve the de- 
sired performance without requiring any statistical knowledge 
of the underlying randomness in the network. 

Most prior performance results for Backpressure are given 
in the following [0(1/V),0(V)] utility-delay tradeoff form 
||6l : Backpressure is able to achieve a utility that is within 
0(1/V) of the optimal utility for any scalar V > 1, while 
guaranteeing a average network delay that is 0(V). Although 
these results provide strong theoretical guarantees for the 
algorithms, the network delay can actually be unsatisfying 
when we achieve a utility that is very close to the optimal, 
i.e., when V is large. 

There have been previous works trying to develop algo- 
rithms that can achieve better utility-delay tradeoffs. Previous 
works J7| and J8) show improved tradeoffs are possible 
for single-hop networks with certain structure, and develops 
optimal [0(l/V),O(log(y))]and [0(1/V),0{VV)] utility- 
delay tradeoffs. However, the algorithms are different from 
basic Backpressure and require knowledge of an "epsilon" 
parameter that measures distance to a performance region 
boundary. Work J51 uses a completely different analytical 
technique to show that similar poly-logarithmic tradeoffs, i.e., 
[0(l/V),0([\og(V)} 2 )], are possible by carefully modify- 
ing the actions taken by the basic Backpressure algorithms. 
However, the algorithm requires a pre-determined learning 
phase, which adds additional complexity to the implemen- 
tation. The current work, following the line of analysis in 
|9l , instead shows that similar poly-logarithmic tradeoffs, 
i.e., [0(1/V), 0([log(y)] 2 )], can be achieved by the orig- 
inal Backpressure algorithm by simply modifying the ser- 
vice discipline from First-in-First-Out (FIFO) to Last-In-First- 
Out (LIFO) (called LIFO-Backpressure below). This is a 
remarkable feature that distinguishes LIFO-Backpressure from 
previous algorithms in Q (8) (9), and provides a deeper 
understanding of backpressure itself, and the role of queue 
backlogs as Lagrange multipliers (see also 13). However, 
this performance improvement is not for free: We must drop 
a small fraction of packets in order to dramatically improve 
delay for the remaining ones. We prove that as the V parameter 
is increased, the fraction of dropped packets quickly converges 
to zero, while maintaining 0(1/V) close-to-optimal utilitiy 
and (9([log(y)] 2 ) average backlog. This provides an analytical 
justification for experimental observations in [ 1 1 that shows a 
related LIFO-Backpressure rule serves up to 98% of the traffic 
with delay that is improved by 2 orders of magnitude. 
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LIFO-Backpressure was proposed in recent empirical work 
ifTol . The authors developed a practical implementation of 
backpressure routing and showed experimentally that applying 
LIFO queuing discipline drastically improves average packet 
delay, but did not provide theoretical guarantees. Another 
notable recent work providing an alternative delay solution is 
ifTTl . which describes a novel backpressure-based per-packet 
randomized routing framework that runs atop the shadow 
queue structure of fTZl while minimizing hop count as ex- 
plored in [ 1 3 1 . Their techniques reduce delay drastically and 
eliminates the per-destination queue complexity, but does not 
provide 0([log(V)] 2 ) average delay guarantees. 

Our analysis of the delay performance of LIFO- 
Backpressure is based on the recent "exponential attraction" 
result developed in J9j. The proof idea can be intuitively 
explained by Fig.[TJ which depicts a simulated backlog process 
of a single queue system with unit packet size under Backpres- 
sure. The left figure demonstrates the "exponential attraction" 
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Fig. 1. The LIFO-Backpressure Idea 

result in (9), which states that queue sizes under Backpressure 
deviate from some fixed point with probability that decreases 
exponentially in the deviation distance. Hence the queue size 
will mostly fluctuate within the interval [Qlow> Qmgh] which 
can be shown to be of (3([log(V)] 2 ) size. This result holds 
under both FIFO and LIFO, as they result in the same queue 
process. Now suppose LIFO is used in this queue. Then from 
the right figure, we see that most of the packets will arrive at 
the queue when the queue size is between Qlow an d Qfiigh, 
and these new packets will always be placed on the top of 
the queue due to the LIFO discipline. Most packets thus enter 
and leave the queue when the queue size is between Q l ow and 
Qmgh- Therefore, these packets "see" a queue with average 
size no more than Qm g h — Qlow — 0([log(V)] 2 ). Now let A 
be the packet arrival rate into the queue, and let A be the 
arrival rate of packets entering when the queue size is in 
[Q Low , Qiiigh] and that eventually depart. Because packets 
always occupy the same buffer slot under LIFO, we see that 
these packets can occupy at most Qmgh — Qlow + 5 max 
buffer slots, ranging from Q Low to Qmgh + S max , where 
Smax = ©(I) is the maximum number of packets that can en- 
ter the queue at any time. We can now apply Little's Theorem 
OH to the buffer slots in the interval [Qlow, Qmgh + <Ws]> 
and we see that average delay for these packets that arrive 
when the queue size is in [Q Low, Q High] satisfies: 



Finally, the exponential attraction result implies that A ps A. 
Hence for almost all packets entering the queue, the average 
delay is D = 0([log(F)] 2 /A). 

This paper is organized as follows. In Section [II] we set up 
our notations. We then present our system model in Section 
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review the Backpressure algorithm in Section [V] The delay 
performance of LIFO-Backpressure is presented in Section fvT| 
Simulation results are presented in Section |VII| We then also 
present experimental testbed results in Section VIII Finally, 
we comment on optimizing a function of time averages in 
Section |IX] 

II. Notations 

Here we first set up the notations used in this paper: K 
represents the set of real numbers. R + (or K_) denotes the 
set of nonnegative (or non-positive) real numbers. R ra (or R" ) 
is the set of n dimensional column vectors, with each element 
being in K (or R+). bold symbols a and a T represent column 
vector and its transpose, a > b means vector a is entrywise 
no less than vector b. \\a — b\ \ is the Euclidean distance of a 
and b. and 1 denote column vector with all elements being 
and 1. [a} + = max[a, 0] and log(-) is the natural log. 

III. System Model 

In this section, we specify the general network model we 
use. We consider a network controller that operates a network 
with the goal of minimizing the time average cost, subject 
to the queue stability constraint. The network is assumed to 
operate in slotted time, i.e., t € {0, 1, 2, ...}. We assume there 
are r > 1 queues in the network. 

A. Network State 

In every slot t, we use S(t) to denote the current net- 
work state, which indicates the current network parameters, 
such as a vector of channel conditions for each link, or 
a collection of other relevant information about the current 
network channels and arrivals. We assume that S(t) evolves 
according a finite state irreducible and aperiodic Markov chain, 
with a total of M different random network states denoted 
as S — {si, S2, ■ ■ ■ , sm}- Let ir s . denote the steady state 
probability of being in state s s . It is easy to see in this case that 
n Si > for all S{. The network controller can observe S(t) 
at the beginning of every slot t, but the ir Si and transition 
probabilities are not necessarily known. 

B. The Cost, Traffic, and Service 

At each time t, after observing S(t) — Sj, the controller 
chooses an action x(t) from a set X^ Si \ i.e., x(t) = x^ Si ^ for 
some e X( 3 *\ The set is called the feasible action 
set for network state s, and is assumed to be time-invariant and 
compact for all s, G S. The cost, traffic, and service generated 
by the chosen action x{t) — x^ Si ^ are as follows: 
(a) The chosen action has an associated cost given by the 
cost function f(t) = f(s i ,x ( - Si '>) : h-> R + (or 

X^ 8 *' i — ^ M_ in reward maximization problems); 



(b) The amount of traffic generated by the action to 
queue j is determined by the traffic function Aj(t) — 
Aj(si,x^) : X W i — ^ K + , in units of packets; 

(c) The amount of service allocated to queue j is given by 
the rate function fij(t) = ^j(si, x^^) : X^ Si > n- R + , in 
units of packets; 

Note that Aj(t) includes both the exogenous arrivals from 
outside the network to queue j, and the endogenous arrivals 
from other queues, i.e., the transmitted packets from other 
queues, to queue j. We assume the functions f(si, •), Hj(si, •) 
and Aj(si, •) are continuous, time-invariant, their magnitudes 
are uniformly upper bounded by some constant 5 max £ (0, oo) 
for all Si, j, and they are known to the network operator. We 
also assume that there exists a set of actions {x t - Si " lk }iZ 1 1 ' 2 ''" A/ f° 
with x( Si ^ k € X^ 3 ^ and some variables $i s ^ > for all Si 
and k with J2k ^fc = 1 f° r a ^ s i> sucn mat 

E' r -{E4 ,() ^( s i. i[( " ) Vw('i.» Wi )]}<-i. ( 2 ) 

for some ?y > for all j. That is, the stability constraints 
are feasible with ^-slackness. Thus, there exists a stationary 
randomized policy that stabilizes all queues (where i?j. s *^ 
represents the probability of choosing action x^ Si ^ k when 

S{t) = Si ) ©. 

C. Queueing, Average Cost, and the Stochastic Problem 

Let q(t) = { qi {t),...,q r {t)) T G R r + , t = 0,1,2,... be 
the queue backlog vector process of the network, in units of 
packets. We assume the following queueing dynamics: 

q 3 {t + 1) = max [q 3 (t) - fij (t) , 0] + Aj (i) Vj, (3) 

and q(Q) = 0. By using ([3]), we assume that when a queue does 
not have enough packets to send, null packets are transmitted. 
In this paper, we adopt the following notion of queue stability: 

r ^ t — 1 r 

E {E^} =limsup- (r)} <oo. (4) 

j=l t_>0 ° r=0j = l 

We also use /~, to denote the time average cost induced by 
an action-choosing policy II, defined as: 

/ a n „^limsup^E{/ n (r)}, (5) 

r— 

where /^,(r) is the cost incurred at time r by policy II. 
We call an action-choosing policy feasible if at every time 
slot t it only chooses actions from the feasible action set 
X\ s w), We then call a feasible action-choosing policy under 
which Q holds a stable policy, and use f* v to denote the 
optimal time average cost over all stable policies. In every 
slot, the network controller observes the current network state 
and chooses a control action, with the goal of minimizing the 
time average cost subject to network stability. This goal can 
be mathematically stated as: (PI) min : f™ v , s.t. (jJJ). In 
the following, we will refer to (PI) as the stochastic problem. 

Note that in some network optimization problems, e.g., 
[15 1, the objective of the network controller is to optimize 
a function of a time average metric. In this case, we see 
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that the Backpressure algorithm and the deterministic problem 
presented in the next section can similarly be constructed, but 
will be slightly different. We will discuss these problems in 
Section E] 

IV. An example of our model 

Here we provide an example to illustrate our model. Con- 
sider the 2-queue network in Fig. [2] In every slot, the network 
operator decides whether or not to allocate one unit of power 
to serve packets at each queue, so as to support all arriving 
traffic, i.e., maintain queue stability, with minimum energy 
expenditure. We assume the network state S(t), which is 
the quadruple {R 1 (t),R 2 {t),CHi(t),CH 2 {t)), evolves ac- 
cording to the finite state Markov chain with three states 
Sl = (l,l,G,S),s 2 = (1, 1,G,G), and s 3 = (0, 0, B, G). 
Here Ri(t) denotes the number of exogenous packet arrivals 
to queue i at time t, and CHi(t) is the state of channel i. 
Ri (t) = x implies that there are x number of packets arriving 
at queue i at time t. CHj(t) = G/B means that channel i 
has a "Good" or "Bad" state. When a link's channel state is 
"Good", one unit of power can serve 2 packets over the link, 
otherwise it can only serve one. We assume power can be 
allocated to both channels without affecting each other. 




Fig. 2. A two queue tandem example. 

In this case, we see that there are three possible network 
states. At each state Sj, the action x^ Si ^ is a pair (£1,0:2), 
with Xi being the amount of energy spent at queue i, 
and (x 1 ,x 2 ) € = {0/1,0/1}. The cost function is 

f(si,x^) = x± + x-i, for all s,. The network states, the 
traffic functions, and the service rate functions are summarized 
in Fig. [3] Note here A\(t) = R\{t) is part of S(t) and is inde- 
pendent of x^ Si ^; while A%{t) — fii(t) +i?2(i) hence depends 
on x^ Si K Also note that A2(t) equals ^i{t) + R2(t) instead 
of mm[ni(t) , qi(t)] due to our idle fill assumption in 

Section lITFCl 
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Fig. 3. The traffic and service functions under different states. 

V. Backpressure and the Deterministic Problem 

In this section, we first review the Backpressure algorithm 
|6| for solving the stochastic problem. Then we define the 
deterministic problem and its dual. We first recall the Back- 
pressure algorithm for utility optimization problems [6|. 

Backpressure: At every time slot t, observe the current 
network state S(t) and the backlog q(t). If S(t) = Sj, choose 
x ( s i) £ ^"( s i) that solves the following: 
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max : -Vf(si,x) + s ^q j {t)[n j (s i ,x) ~ Aj(si,x)] (6) 

3=1 

s.t. 

Depending on the problem structure, (|6]l can usually be 
decomposed into separate parts that are easier to solve, e.g., 
[3 1, [4]. Also, when the network state process S(t) is i.i.d., it 
has been shown in Q that, 



fZ = fav + 0{l/V), q^=0(V), (7) 

where f^ v p and q BP are the expected average cost and the 
expected average network backlog size under Backpressure, 
respectively. When S(t) is Markovian, [3| and [4] show that 
Backpressure achieves an [0(\og(V)/V), 0(V)] utility-delay 
tradeoff if the queue sizes are deterministically upper bounded 
by <d(V) for all time. Without this deterministic backlog 
bound, it has recently been shown that Backpressure achieves 



an [0(( 



-),0(V)] tradeoff under Markovian S(t), with e 



and T t representing the proximity to the optimal utility value 
and the "convergence time" of the Backpressure algorithm 
to that proximity [16|. However, there has not been any 
formal proof that shows the exact [0(1/V),0(V)] utility- 
delay tradeoff of Backpressure under a Markovian S(t). 
We also recall the deterministic problem defined in |9l : 



,( S< 



(8) 



s.t. Aj^^^^Ajisux^) 

Si 

Si 

x (^) e X {Si) Vi = 1,2,..., M, 

where tt s . corresponds to the steady state probability of SYi) = 
Si and x = (x^ Sl \ x^ SM ^) T . The dual problem of (|8b can 
be obtained as follows: 

max : 5(7), s.t. 7^0, (9) 

where 3(7) is called the dual function and is defined as: 



(10) 



E 

i 



7i [^ i (si,x (si ))-/i i (s i ,^ s * ) )] 



Here 7 = (7 1; ...,7 r ) T is the Lagrange multiplier of 
(|8j. It is well known that (7(7) in ( [To| is concave in the 
vector 7, and hence the problem ([9]) can usually be solved 
efficiently, particularly when cost functions and rate functions 
are separable over different network components. Below, we 
use -jy = (7v-i,7v2) ■•■'7yr ) T to denote an optimal solution 
of the problem (|9|l with the corresponding V. 

VI. Performance of LIFO Backpressure 

In this section, we analyze the performance of Back- 
pressure with the LIFO queueing discipline (called LIFO- 
Backpressure). The idea of using LIFO under Backpressure 
is first proposed in iTTOl . although they did not provide any 



theoretical performance guarantee. We will show, under some 
mild conditions (to be stated in Theorem [31, that under LIFO- 
Backpressure, the time average delay for almost all packets 
entering the network is 0([log(V)] 2 ) when the utility is 
pushed to within 0(1/V) of the optimal value. Note that the 
implementation complexity of LIFO-Backpressure is the same 
as the original Backpressure, and LIFO-Backpressure only 
requires the knowledge of the instantaneous network condi- 
tion. This is a remarkable feature that distinguishes it from 
the previous algorithms achieving similar poly-logarithmic 
tradeoffs in the i.i.d. case, e.g., |7| |8| [9|, which all require 
knowledge of some implicit network parameters other than the 
instant network state. Below, we first provide a simple example 
to demonstrate the need for careful treatment of the usage of 
LIFO in Backpressure algorithms, and then present a modified 
Little's theorem that will be used for our proof. 

A. A simple example on the LIFO delay 

Consider a slotted system where two packets arrive at time 
0, and one packet periodically arrives every slot thereafter (at 
times 1, 2,3,.. .). The system is initially empty and can serve 
exactly one packet per slot. The arrival rate A is clearly 1 
packet/slot (so that A = 1). Further, under either FIFO or 
LIFO service, there are always 2 packets in the system, so 
Q = 2. 

Under FIFO service, the first packet has a delay of 1 and 
all packets thereafter have a delay of 2: 

W FIFO = 1 , Wf IFO = 2 Vz E {2, 3,4,.. .}, 

where Wf IF is the delay of the i th packet under FIFO 



LIFO 



is similarly defined for LIFO). We thus have: 



W FIF °^ lim ±Ywf IFO = 2. 

i=l 



K 



K- 



FIFO — FIFO — 

Thus, XW = 1 x 2 = 2, Q = 2, and so XW = Q 
indeed holds. 

Now consider the same system under LIFO service. We still 
have X = 1, Q = 2. However, in this case the first packet never 
departs, while all other packets have a delay equal to 1 slot: 

Wt IFO = 00 , Wt IFO = 1 Vi € {2, 3, 4, . . .}. 

Thus, for all integers K > 0: 

K 



LIFO 



K 

i=i 

and so W LIF ° = 00. Clearly XW LIF ° 7^ Q. On the other 
hand, if we ignore the one packet with infinite delay, we note 
that all other packets get a delay of 1 (exactly half the delay 
in the FIFO system). Thus, in this example, LIFO service 
significantly improves delay for all but the first packet. 

For the above LIFO example, it is interesting to note that 
if we define Q and W as the average backlog and delay 
associated only with those packets that eventually depart, then 
we have Q = 1, W = 1, and the equation XW = Q indeed 
holds. This motivates the theorem in the next subsection, 
which considers a time average only over those packets that 
eventually depart. 
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B. A Modified Little 's Theorem for LIFO systems 

We now present the modified Little's theorem. Let B rep- 
resent a finite set of buffer locations for a LIFO queueing 
system. Let N(i) be the number of arrivals that use a buffer 
location within set B up to time t. Let D(t) be the number of 
departures from a buffer location within the set B up to time 
t. Let Wi be the delay of the zth job to depart from the set 
B. Define W as the lira sup average delay considering only 
those jobs that depart: 



W= lim sup 



1 

W) 



D(t) 

E 

i=l 



> such 



We then have the following theorem: 

Theorem 1: Suppose there is a constant \ r 
that with probability 1: 

liminf > X min , 

Further suppose that lim^oo D(t) — oo with probability 1 (so 
the number of departures is infinite). Then the average delay 
W satisfies: 



W= lim sup 



1 

W) 



D(t) 

E 



Wi < \B\/K. 



where \B\ is the size of the finite set B. 

Proof: See Appendix A. ■ 

C. LIFO-Backpressure Proof 

We now provide the analysis of LIFO-Backpressure. To 
prove our result, we first have the following theorem, which 
is the first to show that Backpressure (with either FIFO or 
LIFO) achieves the exact [0(1/V),0(V)] utility-delay trade- 
off under a Markovian network state process. It generalizes 
the [0(1/V),0(V)} performance result of Backpressure in 
the i.i.d. case in Q. 

Theorem 2: Suppose S(t) is a finite state irreducible and 
aperiodic Markov chaiij^] and condition ^ holds, Backpres- 
sure (with either FIFO or LIFO) achieves the following: 



C P = f: v + 0{l/V), q^=0(V) 



BP 



(ID 



where f^ v p and q BP are the expected time average cost and 
backlog under Backpressure. 

Proof: See 02). ■ 

Theorem [2] thus shows that LIFO-Backpressure guarantees 
an average backlog of 0(V) when pushing the utility to within 
0(1/V) of the optimal value. We now consider the delay 
performance of LIFO-Backpressure. For our analysis, we need 
the following theorem (which is Theorem 1 in [9|). 

Theorem 3: Suppose that ~fy is unique, that the slackness 
condition (|2| holds, and that the dual function g(-y) satisfies: 



ghv) > 9(1) + L\hv - 1\\ v 7 _-o, 



(12) 



for some constant L > independent of V. Then un- 
der Backpressure with FIFO or LIFO, there exist constants 

2 In 1171 . the theorem is proven under more general Markovian S(t) 
processes that include the S(t) process assumed here. 



D, K, c* — 0(1), i.e., all independent of V, such that for any 

m € K+, 



V {r \D,Km) < c*e~ m , 



(13) 



where V^^D^Km) is defined: 

V {r) (D,Km,) (14) 
1 * _1 

= lim sup - ^2 Pr i^h kj( T ) ~lvj\ > D + Km). 

1 r=0 

Proof: See (3. ■ 
Note that if a steady state distribution exists for q(t), e.g., 
when all queue sizes are integers, then V^(D, Km) is indeed 
the steady state probability that there exists a queue j whose 
queue value deviates from 7^ by more than D+Km distance. 
In this case, Theorem [3j states that qj(t) deviates from 7^. by 
0(log(V)) distance with probability 0(l/V). Hence when 
V is large, qj(t) will mostly be within 0(log(V)) distance 
from jyj . Also note that the conditions of Theorem [3j are not 
very restrictive. The condition ( fT2} can usually be satisfied in 
practice when the action space is finite, in which case the dual 
function g(j) is polyhedral (see [9| for more discussions). The 
uniqueness of j v can usually be satisfied in many network 
utility optimization problems, e.g., G)- 

We now present the main result of this paper with respect 
to the delay performance of LIFO-Backpressure. Below, the 
notion "average arrival rate" is defined as follows: Let Aj(t) 
be the number of packets entering queue j at time t. Then the 
time average arrival rate of these packets is defined (assuming 
it exists): Xj = lim^oo | St=o ^0 '( T )- For the theorem, 
we assume that time averages under Backpressure exist with 
probability 1. This is a reasonable assumption, and holds 
whenever the resulting discrete time Markov chain for the 
queue vector q(t) under backpressure is countably infinite 
and irreducible. Note that the state space is indeed countably 
infinite if we assume packets take integer units. If the system 
is also irreducible then the finite average backlog result of 
Theorem [2] implies that all states are positive recurrent. 

Let D,K,c* be constants as defined in Theorem [5] and 
recall that these are 0(1) (independent of V). Assume V > 1, 
and define Qj,m g h and Q jiLow as: 



Qj^High 



7vj +D + K[\og(V)} 



Q j<Low 4 max[j vj -D-K[log(V)] 2 ,0]. 

Define the interval BjA[Qj t High> Qj,Low]- The following the- 
orem considers the rate and delay of packets that enter when 
<7j (i) € Bj and that eventually depart. 

Theorem 4: Suppose that V > 1, that 7^ is unique, that 
the slackness assumption (|2| holds, and that the dual function 
5(7) satisfies: 



l(l*v) > 9(l) + L\\lv - l\\ V 7 _-0, 



(15) 



for some constant L > independent of V. Define D,K,c* 
as in Theorem [3] and define Bj as above. Then for any queue 
j with a time average input rate Xj > 0, we have under LIFO- 
Backpressure that: 
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(a) The rate Aj of packets that both arrive to queue j when 
Qj{t) £ Bj and that eventually depart the queue satisfies: 

+ 



A j > Aj > 



A; 



(16) 



(b) The average delay of these packets is at most Wb OU nd, 
where: 



ound z 



,[2D + 2K[log(V)f 



This theorem says that the delay of packets that enter when 
£ Bj and that eventually depart is at most 0([log(V)] 2 ). 
Further, by (16) , when V is large, these packets represent the 
overwhelming majority, in that the rate of packets not in this 
set is at most 0{l/V Xo ^). 

Proof: (Theorem [4|i Theorem [2] shows that average queue 
backlog is finite. Thus, there can be at most a finite number 
of packets that enter the queue and never depart, so the rate of 
packets arriving that never depart must be 0. It follows that Aj 
is equal to the rate at which packets arrive when qj(t) £ Bj. 
Define the indicator function lj(t) to be 1 if qj(t) ^ Bj, and 



else. Define A^Aj 



Aj. Then with probability 1 we get: [^j 



= Hmi|>(r)l;(T) 

T=Q 
00 T=0 

Then using the fact that Aj (t) < 5 max for all j, t: 

EiAjWlj®} = ^{A^q^iB^Priq^iBj) 
< 5 max Pr(qj(t) g [Q -j, low, Q 3 , High])- 

Therefore: 

1 

K < 5 max lim - Y] Pr(qj{r) £ [Qj , L ow,Qj ,m g h]) 



T = 
t-1 



< 



lim \ J2 Pr (ki(r) - 7vy| > D + Km), 



r=0 



where we define m=[log(V)] 2 , and note that m > because 
V > 1. From Theorem [3] we thus have: 

" (17) 



< X? < 5 n 



V l °s(v) ' 



This completes the proof of part (a). Now define Bj = 

=[Qj 

,High ~t~ $7nax-> Qj,Low\- Since Bj C Bj,_we, see that the 
rate of the packets that enter Bj is at least Aj. Part (b) then 
follows from Theorem [T] and the facts that queue j is stable 
and that \B } \ <2D + 2K [log(V)] 2 + S max . ■ 
Note that if Aj = 6(1), we see from Theorem [4] that, under 
LIFO-Backpressure, the time average delay for almost all 
packets going through queue j is only (9([log(t/)] 2 ). Applying 
this argument to all network queues with 0(1) input rates, we 
see that all but a tiny fraction of the traffic entering the network 
only experiences a delay of 0([log(V^)] 2 ). This contrasts with 

3 The time average expectation is the same as the pure time average by 
the Lebesgue Dominated Convergence Theorem, because we assume the pure 
time average exists with probability 1, and that < Aj(t) < 8 ma x Vi. 



the delay performance result of the usual Backpressure with 
FIFO, which states that the time average delay will be Q(V) 
for all packets [9|. Also note that under LIFO-Backpressure, 
some packets may stay in the queue for very long time. 
This problem can be compensated by introducing certain 
coding techniques, e.g., fountain codes QjD , into the LIFO- 
Backpressure algorithm. 

VII. Simulation 

In this section, we provide simulation results of the LIFO- 
Backpressure algorithm. We consider the network shown in 
Fig. |4j where we try to support a flow sourced by Node 1 
destined for Node 7 with minimum energy consumption. 



A(t). 




Fig. 4. A multihop network, (a, b) represents the HIGH probability a and 
the rate b obtained with one unit of power when HIGH. 

We assume that A(t) evolves according to the 2-state 
Markov chain in Fig. [5] When the state is HIGH, A(t) = 3, 
else A(t) = 0. We assume that the condition of each link 
can either be HIGH or LOW at a time. All the links except 
link (2,4) and link (6,7) are assumed to be i.i.d. every time 
slot, whereas the conditions of link (2,4) and link (6,7) 
are assumed to be evolving according to independent 2-state 
Markov chains in Fig. [5] Each link's HIGH probability and 
unit power rate at the HIGH state is shown in Fig. [4] The 
unit power rates of the links at the LOW state are all assumed 
to be 1. We assume that the link states are all independent and 
there is no interference. However, each node can only spend 
one unit of power per slot to transmit over one outgoing link, 
although it can simultaneously receive from multiple incoming 
links. The goal is to minimize the time average power while 
maintaining network stability. 



0.7 j HIGH ) ( LOW ) 0.7 



Fig. 5. The two state Markov chain with the transition probabilities. 

We simulate Backpressure with both LIFO and FIFO for 
10 6 slots with V £ {20, 50, 100, 200, 500}. It can be verified 
that the backlog vector converges to a unique attractor as V 
increases in this case. The left two plots in Fig. [6] show the 
average power consumption and the average backlog under 
LIFO-Backpressure. It can be observed that the average power 
quickly converges to the optimal value and that the average 
backlog grows linearly in V. The right plot of Fig. [6] shows 
the percentage of time when there exists a qj whose value 
deviates from -fy- by more than 2[log(T^)] 2 . As we can see, 
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this percentage is always very small, i.e., between 0.002 and 
0.013, showing a good match between the theory and the 
simulation results. 





Fig. 6. LEFT: average network power consumption. MIDDLE: average 
network backlog size. RIGHT: percentage of time when 3 qj such that 

k-7£,-l>2[log(V)] 2 . 



Fig. [7] compares the delay statistics of LIFO and FIFO for 
more than 99.9% of the packets that leave the system before 
the simulation ends, under the cases V = 100 and V = 500. 
We see that LIFO not only dramatically reduces the average 
packet delay for these packets, but also greatly reduces the 
delay for most of these packets. For instance, when V — 500, 
under FIFO, almost all packets experience the average delay 
around 1220 slots. Whereas under LIFO, the average packet 
delay is brought down to 78. Moreover, 52.9% of the packets 
only experience delay less than 20 slots, and 90.4% of the 
packets experience delay less than 100 slots. Hence most 
packets' delay are reduced by a factor of 12 under LIFO as 
compared to that under FIFO! 



V=100 
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Avg. DL 


% DL < 20 


% DL < 50 


% DL < 100 


LIFO 


55.4 


55.0 


82.1 
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FIFO 
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Case 
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% DL < 20 
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FIFO 


1219.8 












Fig. 7. Delay statistics under Backpressure with LIFO and FIFO for packets 
that leave the system before simulation ends (more than 99.9%). %DL < a 
is the percentage of packets that enter the network and has delay less than a. 

Fig. [8] also shows the delay for the first 20000 packets that 
enter the network in the case when V = 500. We see that under 
Backpressure plus LIFO, most of the packets experience very 
small delay; while under Backpressure with FIFO, each packet 
experiences roughly the average delay. 

VIII. Empirical Validation 

In this section we validate our analysis empirically by carry- 
ing out new experiments over the same testbed and Backpres- 
sure Collection Protocol (BCP) code of iflOl . This prior work 
did not empirically evaluate the relationship between V, finite 
storage availability, packet latency and packet discard rate. We 
note that BCP runs atop the default CSMA MAC for TinyOS 
which is not known to be throughput optimal, that the testbed 
may not precisely be defined by a finite state Markovian 



Fig. 



Packet Delay under Backpressure with LIFO and FIFO 



evolution, and finally that limited storage availability on real 
wireless sensor nodes mandates the introduction of virtual 
queues to maintain backpressure values in the presence of data 
queue overflows. 

In order to avoid using very large data buffers, in IflOl the 
forwarding queue of BCP has been implemented as a floating 
queue. The concept of a floating queue is shown in Figure 



10 which operates with a finite data queue of size D max 
residing atop a virtual queue which preserves backpressure 
levels. Packets that arrive to a full data queue result in a 
data queue discard and the incrementing of the underlying 
virtual queue counter. Underflow events (in which a virtual 
backlog exists but the data queue is empty) results in null 
packet generation, which are filtered and then discarded by 
the destination. 

Despite these real-world differences, we are able to demon- 
strate clear order-equivalent delay gains due to LIFO usage in 
BCP in the following experimentation. 




40 39,A 37 36 3B 34 33 i 



1 



28 27 ~ p 2 



26 2 ^E^ 24 23 22 



Fig. 9. The 40 tMote Sky devices used in experimentation on Tutornet. 

A. Testbed and General Setup 

To demonstrate the empirical results, we deployed a col- 
lection scenario across 40 nodes within the Tutornet testbed 
(see Figure [9j. This deployment consisted of Tmote Sky 
devices embedded in the 4th floor of Ronald Tutor Hall at 
the University of Southern California. 

In these experiments, one sink mote (ID 1 in Figure [9]) 
was designated and the remaining 39 motes sourced traffic 
simultaneously, to be collected at the sink. The Tmote Sky 
devices were programmed to operate on 802.15.4 channel 26, 
selected for the low external interference in this spectrum on 
Tutornet. Further, the motes were programmed to transmit 
at -15 dBm to provide reasonable interconnectivity. These 
experimental settings are identical to those used in [10|. 
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time j jj 



IV V VI VII 



Fig. 10. The floating LIFO queues of 1 10] drop from the data 
queue during overflow, placing the discards within an underlying 
virtual queue. Services that cause data queue underflows generate 
null packets, reducing the virtual queue size. 



We vary D max over experimentation. In practice, BCP 
defaults to a D max setting of 12 packets, the maximum 
reasonable resource allocation for a packet forwarding queue 
in these highly constrained devices. 

B. Experiment Parameters 

Experiments consisted of Poisson traffic at 1.0 packets 
per second per source for a duration of 20 minutes. This 
source load is moderately high, as the boundary of the ca- 
pacity region for BCP running on this subset of motes has 
previously been documented at 1.6 packets per second per 
source ifTol . A total of 36 experiments were run using the 
standard BCP LIFO queue mechanism, for all combinations 
of V e {1,2,3,4,6,8,10,12} and LIFO storage threshold 
D m ax £ {2, 4, 8, 12}. In order to present a delay baseline for 
Backpressure we additionally modified the BCP source code 
and ran experiments with 32-packet FIFO queues (no floating 
queues) for V £ {1,2,3}. Q 

C. Results 



Testbed results in Figure 11 provide the system average 

/;</,:, and 



packet delay from source to sink over V and D 
includes 95% confidence intervals. Delay in our FIFO imple- 
mentation scales linearly with V, as predicted by the analysis 
in (9). This yields an average delay that grows very rapidly 
with V, already greater than 9 seconds per packet for V = 3. 
Meanwhile, the LIFO floating queue of BCP performs much 
differently. We have plotted a scaled [log(V)] 2 target, and 
note that as D max increases the average packet delay remains 
bounded by 9([log(^)] 2 ). 

These delay gains are only possible as a result of discards 
made by the LIFO floating queue mechanism that occur when 
the queue size fluctuates beyond the capability of the finite 



data queue to smooth. Figure 12 gives the system packet 
loss rate of BCP's LIFO floating queue mechanism over V. 
Note that the poly-logarithmic delay performance of Figure 
[TT| is achieved even for data queue size 12, which itself 

4 These relatively small V values are due to the constraint that the motes 
have small data buffers. Using larger V values will cause buffer overflow at 
the motes. 
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Fig. 11. System average source to sink packet delay for BCP FIFO 
versus BCP LIFO implementation over various V parameter settings. 



drops at most 5% of traffic at V = 12. We cannot state 
conclusively from these results that the drop rate scales like 
0( y aQ ii g (v) )■ We hypothesize that a larger V value would be 
required in order to observe the predicted drop rate scaling. 
Bringing these results back to real-world implications, note 
that BCP (which minimizes a penalty function of packet 
retransmissions) performs very poorly with V = 0, and was 
found to have minimal penalty improvement for V greater 
than 2. At this low V value, BCP's 12-packet forwarding 
queue demonstrates zero packet drops in the results presented 
here. These experiments, combined with those of ifTUll suggest 
strongly that the drop rate scaling may be inconsequential in 
many real world applications. 

Packet Discard Percentage Versus V For Various LIFO Size 
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Fig. 12. System packet loss rate of BCP LIFO implementation over 
various V parameter settings. 

In order to explore the queue backlog characteristics and 



compare with our analysis, Figure 13 presents a histogram 
of queue backlog frequency for rear-network-node 38 over 
various V settings. This node was observed to have the worst 
queue size fluctuations among all thirty-nine sources. For 
V — 2, the queue backlog is very sharply distributed and 
fluctuates outside the range [11 — 15] only 5.92% of the 
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experiment. As V is increased, the queue attraction is evident. 
For V = 8 we find that the queue deviates outside the range 
[41 — 54] only 5.41% of the experiment. The queue deviation 
is clearly scaling sub-linearly, as a four-fold increase in V 
required only a 2.8 fold increase in D max for comparable 
drop performance. 
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Fig. 13. Histogram of queue backlog frequency for rear-network-node 
38 over various V settings. 

IX. Optimizing Functions of Time averages 

So far we have focused on optimizing time averages of 
functions, we now consider the case when the objective of 
the network controller is to optimize a function of some 
time average metric, e.g., lfl5l . Specifically, we assume that 
the action x(t) at time t incurs some instantaneous network 
attribute vector y(t) = y{x(t)) = (yi(t), y K (t)) T £ R+, 
and the objective of the network controller is to minimize 
a cost function Cost(y(t)) : Rf -> R+, where y(t) 
represents the time average value of y(t). We assume that 
the function Cost(-) is continuous, convex and is component- 
wise increasing, and that \y k (x(t))\ < S max for all k, x(t). In 
this case, we see that the Backpressure algorithm in Section W\ 
cannot be directly applied and the deterministic problem (|8F 
also needs to be modified. 

To tackle this problem using the Backpressure algorithm, we 
introduce an auxiliary vector z(t) — (zi(t), ZK{t)) T ■ We 
then define the following virtual queues H k (t),j = 1,...,K 
that evolves as follows: 



H k (t + 1) = max [H k {t) - y k (t),0] + z k (t). 



(18) 



These virtual queues are introduced for ensuring that the 
average value of y k (t) is no less than the average value of 
z k (t). We will then try to maximize the time average of 
the function Cost(z(t)), subject to the constraint that the 
actual queues qj(t) and the virtual queues H k (t) must all 
be stable. Specifically, the Backpressure algorithm for this 
problem works as follows: 

5 The case for maximizing a utility function of long term averages can be 
treated in a similar way. 



Backpressure: At every time slot t, observe the current 
network state S(t), and the backlogs q(t) and H(t). If 
S(t) = Si, do the following: 

1) Auxiliary vector: choose the vector z(t) by solving: 

min : VCost(z) + ^ H k (t)z k 

k 

s.t. 0<z k <5 

max • (19) 

2) Action: choose the action x(t) £ A?( Si ) that solves: 



mux: > \ H k (t)yk(x) + qj(t)[fj l j(s i , x) - Aj(sj, x)] 

3 



S.t. X £ X {si) . 



(20) 



In this case, one can also show that this Backpressure al- 
gorithm achieves the [0(1/V),0(V)] utility-delay tradeoff 
under a Markovian S(t) process. We also note that in this 
case, the deterministic problem is slightly different. Indeed, 
the intuitively formulation will be of the following form: 



min : T{x) = VCost(^ 7r Si y(x (Sl) )) 



(21) 



s.t. AjW^Y.Ks.Ajis^x^) 

Si 

Si 

x (»*)eX M Vi = 1,2,..., M. 
However, the dual problem of this optimization problem is 



not separable, i.e., not of the form of ( 10 1, unless the function 
Cost(-) is linear or if there exists an optimal action that is in 
every feasible action set X^ Si \ e.g., [15|. To get rid of this 
problem, we introduce the auxiliary vector z = (z%, zk) T 
and change the problem to: 



mm : 
s.t. 



F{x) = VCosi{z) 



(22) 



Si 

<B j (x)± , $2n ai iJ lj (s i ,xM) Vj 

Si 

x i^) e x (s ^ Mi = 1,2,.. .,M. 
It can be shown that this modified problem is equivalent to 



(21 1. Now we see that it is indeed due to the non-separable 



feature of pi) that we need to introduce the auxiliary vector 
z(t) in the Backpressure problem. We also note that the 



problem (22i actually has the form of mj. Therefore, all 
previous results on ([8]), e.g., Theorem [3] and|4] will also apply 
to problem (22 1. 

Appendix A - Proof of[T] 

Here we provide the proof of Theorem [T] 

Proof: Consider a sample path for which the liminf 
arrival rate is at least A m in an d f° r which we have an 
infinite number of departures (this happens with probability 
1 by assumption). There must be a non-empty subset of 
B consisting of buffer locations that experience an infinite 



to 



number of departures. Call this subset B. Now let be 
the delay of the i th departure from 6, let D^ b '(t) denote the 
number of departures from a buffer slot b e B up to time t, 
and use Q^ b '(t) to denote the occupancy of the buffer slot b 
at time t. Note that Q (b) (£) is either or 1. For all t > 0, it 
can be shown that: 



D m (t) 



E w ^ ± i Q {b) ^ 

.•— i Jo 



(23) 



This can be seen from Fig. 14 below. 



(b) 



(b) 



(b) 



(b) 



Fig. 14. An illustration of inequality (23) for a particular buffer location b. 
At time t in the figure, we have fl"(t] = 3. 

Therefore, we have: 



E 

bee 1=1 



< 
< 



|B|dr 



(24) 



The left-hand-side of the above inequality is equal to the 
sum of all delays of jobs that depart from locations in B up to 
time t. All other buffer locations (in B but not in B) experience 
only a finite number of departures. Let J represent an index 
set that indexes all of the (finite number) of jobs that depart 
from these other locations. Note that the delay Wj for each 
job j £ J is finite (because, by definition, job j eventually 
departs). We thus have: 



D(t) £>< b) (t) 



E^<E E wf + E^ 



where the inequality is because the second term on the right- 
hand-side sums over jobs in J , and these jobs may not have 



departed by time t. Combining the above and ( 24 1 yields for 
all t > 0: 

D(t) 



Dividing by D(t) yields: 



1 



D(t) 



\B\t 



1 



m - m m 

Taking a limsup as t — > oo yields: 



E^- 



limsup r 
t ^oo P D{t) f-* 

2 — 1 



2_^Wi< limsup 



(25) 



t— >oo 



where we have used the fact that J2jeJ 
and D(t) —> oo as t — >• oo, so that: 



D(ty 

Wj is a finite number, 



Urn sup -=-rr 
D{t) 



E 



Wj = 0. 



Now note that, because each buffer location in B can hold 
at most one job, the number of departures D(t) is at least 
N(t) — \B\, which is a positive number for sufficiently large 
t. Thus: 



,. \B\t 
hm sup ^. . 
t^oo y D(t) 



< 



< 



lim sup 

t— >oo 

lim sup 

t— >oo 

\B\/\ mi 



\B\t 



N(t) \B\ 



\B\ 



N(t)/t-\B\/t 



Using this in ( |25| ) proves the result. ■ 
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